Self-Driving Car Engineer Nanodegree¶
Deep Learning¶
Project: Build a Traffic Sign Recognition Classifier¶
In this notebook, a template is provided for you to implement your functionality in stages which is required to successfully complete this project. If additional code is required that cannot be included in the notebook, be sure that the Python code is successfully imported and included in your submission, if necessary. Sections that begin with 'Implementation' in the header indicate where you should begin your implementation for your project. Note that some sections of implementation are optional, and will be marked with 'Optional' in the header.
In addition to implementing code, there will be questions that you must answer which relate to the project and your implementation. Each section where you will answer a question is preceded by a 'Question' header. Carefully read each question and provide thorough answers in the following text boxes that begin with 'Answer:'. Your project submission will be evaluated based on your answers to each of the questions and the implementation you provide.
Note: Code and Markdown cells can be executed using the Shift + Enter keyboard shortcut. In addition, Markdown cells can be edited by typically double-clicking the cell to enter edit mode.
Step 1: Dataset Exploration¶
Visualize the German Traffic Signs Dataset. This is open ended, some suggestions include: plotting traffic signs images, plotting the count of each sign, etc. Be creative!
The pickled data is a dictionary with 4 key/value pairs:
- features -> the images pixel values, (width, height, channels)
- labels -> the label of the traffic sign
- sizes -> the original width and height of the image, (width, height)
- coords -> coordinates of a bounding box around the sign in the image, (x1, y1, x2, y2). Based the original image (not the resized version).
In [1]:
import numpy as np
import tensorflow as tf
import cv2
import matplotlib.pyplot as plt
import time
import csv
import itertools
import os
from scipy import ndimage
from tqdm import tnrange
%matplotlib inline
In [2]:
# Load pickled data
import pickle
# TODO: fill this in based on where you saved the training and testing data
training_file = '../../sdc/traffic-signs-data/train.p'
testing_file = '../../sdc/traffic-signs-data/test.p'
with open(training_file, mode='rb') as f:
train = pickle.load(f)
with open(testing_file, mode='rb') as f:
test = pickle.load(f)
X_train, y_train = train['features'], train['labels']
X_test, y_test = test['features'], test['labels']
# Save Original Test for future evaluation
X_test_orig = X_test
y_test_orig = y_test
In [3]:
# Load Sign Names
def read_signnames(file):
with open(file) as f:
csvreader = csv.reader(f)
header = next(csvreader)
signnames = {}
for row in csvreader:
signnames[int(row[0])] = row[1]
return signnames
signnames = read_signnames('signnames.csv')
print("Sign names loaded: len = ", len(signnames))
In [4]:
print('X_train.shape =', X_train.shape)
print('y_train.shape =', y_train.shape)
print('X_test.shape =', X_test.shape)
print('y_test.shape =', y_test.shape)
print('image shape = ', X_train[0].shape)
np.max(y_train)+1
Out[4]:
In [5]:
### To start off let's do a basic data summary.
# TODO: number of training examples
n_train = X_train.shape[0]
# TODO: number of testing examples
n_test = X_test.shape[0]
# TODO: what's the shape of an image?
image_shape = X_train[0].shape
# TODO: how many classes are in the dataset
n_classes = np.max(y_train) + 1
print("Number of training examples =", n_train)
print("Number of testing examples =", n_test)
print("Image data shape =", image_shape)
print("Number of classes =", n_classes)
In [6]:
### Data exploration visualization goes here.
### Feel free to use as many code cells as needed.
# Show all classes
def show_all_classes(data, labels, num_line=9, num_imgs=3):
for c in range(n_classes):
if c * num_imgs % num_line == 0:
plt.figure((c * num_imgs) // num_line, figsize=(18, 2))
s = data[labels == c]
indices = np.random.randint(0, len(s), num_imgs)
for n, idx in enumerate(indices):
plot_num = (c * num_imgs) % num_line + n + 1
plt.subplot(1, num_line, plot_num)
cur_axes = plt.gca()
cur_axes.axes.get_xaxis().set_ticks([])
cur_axes.axes.get_yaxis().set_ticks([])
plt.imshow(s[idx])
signname = signnames[c][:17] + '..' if len(signnames[c]) > 17 else signnames[c]
plt.title('%d - %s' % (c, signname), fontsize=10)
show_all_classes(X_train, y_train)
In [7]:
# Draw the classes distribution in X_train samples
def show_classes_distribution(labels, title = None):
# Count classes samples
uniq_labels = sorted(set(labels.tolist()))
n_labels = len(uniq_labels)
class_counts = np.zeros([n_labels])
for c in uniq_labels:
class_counts[c] = np.sum(labels == c)
y_pos = np.arange(n_labels)
plt.figure(figsize=(18, 5))
plt.bar(uniq_labels, class_counts)
plt.xticks(y_pos)
plt.ylabel('Classes Counts')
if title: plt.title(title)
show_classes_distribution(y_train, title='Train Classes Distribution')
show_classes_distribution(y_test, title='Test Classes Distribution')
Step 2: Design and Test a Model Architecture¶
Design and implement a deep learning model that learns to recognize traffic signs. Train and test your model on the German Traffic Sign Dataset.
There are various aspects to consider when thinking about this problem:
- Your model can be derived from a deep feedforward net or a deep convolutional network.
- Play around preprocessing techniques (normalization, rgb to grayscale, etc)
- Number of examples per label (some have more than others).
- Generate fake data.
Here is an example of a published baseline model on this problem. It's not required to be familiar with the approach used in the paper but, it's good practice to try to read papers like these.
Implementation¶
Use the code cell (or multiple code cells, if necessary) to implement the first step of your project. Once you have completed your implementation and are satisfied with the results, be sure to thoroughly answer the questions that follow.
In [8]:
### Preprocess the data here.
### Feel free to use as many code cells as needed.
def grayscale(img):
"""Applies the Grayscale transform
This will return an image with only one color channel
but NOTE: to see the returned image as grayscale
you should call plt.imshow(gray, cmap='gray')"""
return cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
def data_to_grayscale(data):
# Transform to grayscale
shape = data.shape
data_gray = np.zeros((shape[0], shape[1], shape[2]))
for idx, image in enumerate(data):
data_gray[idx] = grayscale(image)
return data_gray
def random_rotate(img):
angle = np.random.randint(-25, 25)
# print('angle =', angle)
return ndimage.rotate(img, angle, reshape=False)
def random_gaussian_filter(img):
sigma = np.random.choice([0.2,0.5,0.7,1])
# sigma = 0.5
# print('sigma =', sigma)
return ndimage.gaussian_filter(img, sigma=sigma)
def random_noise(img):
noise = np.zeros_like(img)
noise_lvl = np.random.choice([10, 20, 30])
cv2.randn(noise, 0, noise_lvl.flatten())
# print("noise =", noise)
return img + noise
def random_image_transform(img):
transforms = {
0: random_rotate,
1: random_gaussian_filter,
2: random_noise
}
n = np.random.randint(3)
im = transforms[n](img)
# im = cv2.cvtColor(im, cv2.COLOR_BGR2RGB)
return im
"""
X_train_gray = data_to_grayscale(X_train)
# Show random images from X_train_gray
n_samples = 12
n_samples_idx = np.random.randint(0, len(X_train_gray), n_samples)
gray_samples = X_train_gray[n_samples_idx]
plt.figure(figsize=(18, 5))
for n, img in enumerate(gray_samples):
plt.subplot(1, n_samples, n + 1)
cur_axes = plt.gca()
cur_axes.axes.get_xaxis().set_ticks([])
cur_axes.axes.get_yaxis().set_ticks([])
plt.imshow(img, cmap='gray')
"""
Out[8]:
Question 1¶
Describe the techniques used to preprocess the data.
Answer:
In [9]:
# Combine train and test into one dataset
# X = X_train
# y = y_train
X = np.concatenate((X_train, X_test), axis=0)
y = np.concatenate((y_train, y_test), axis=0)
In [10]:
### Generate data additional (if you want to!)
### and split the data into training/validation/testing sets here.
### Feel free to use as many code cells as needed.
# Generate additional data hints
# real-world variabilities such as viewpoint variations, lighting conditions
# (saturations, low-contrast), motion-blur, occlusions, sun
# glare, physical damage, colors fading, graffiti, stickers and
# an input resolution as low as 15x15
def generate_additional_images(X, y):
# Count classes samples
uniq_labels = sorted(set(y.tolist()))
n_labels = len(uniq_labels)
class_counts = np.zeros([n_labels])
all_ind = np.arange(len(y))
img_indices = {}
for c in uniq_labels:
class_counts[c] = np.sum(y == c)
img_indices[c] = all_ind[y == c]
print("len_X =", len(X))
print("len_y =", len(y))
print("class_counts =", class_counts)
max_num = np.max(class_counts)
min_num = np.min(class_counts)
print("max_num =", max_num)
print("min_num =", min_num)
errors = 0
classes_range = tnrange(n_labels, desc="Classes") # tqdm progress
for c in classes_range:
c_num = np.sum(y == c)
num = max_num - c_num
# print('%2d: c_num = %d, num = %d' % (
# c, c_num, num))
nimgs = []
nimgs_labels = []
num_range = tnrange(int(num), desc="C=%d" % c) # tqdm progress
for n in num_range:
# Draw random image of class argmin
sample = np.random.choice(img_indices[c])
# print("sample =", sample)
# Transform it
try:
nimg = random_image_transform(X[sample])
except:
errors += 1
nimg = random_gaussian_filter(X[sample])
# continue
# print('Error! c = %d, sample = %d' % (c, sample))
# Append to X and append to y
nimgs.append(nimg)
nimgs_labels.append(c)
if len(nimgs) > 0:
# print("1: len_X =", len(X))
# print("1: X.shape =", X.shape)
# print("1: len_nimgs =", len(nimgs))
# print("1: nimgs.shape =", nimgs[0].shape)
X = np.append(X, np.array(nimgs), axis=0)
# print("1: len_X =", len(X))
plt.imshow(nimgs[0])
y = np.append(y, nimgs_labels)
class_counts[c] += num
#
# print("all_ind =", img_indices[c])
# print("sample =", np.random.choice(img_indices[c]))
# max_num - min_num > 0.1 * max_num
# add sample of class argmin
# argmin = np.argmin(class_counts)
# print("argmin =", argmin)
# Append to X and append to y
# +1 to class_counts
# max_num = np.max(class_counts)
# min_num = np.min(class_counts)
# print("max_num =", max_num)
# print("min_num =", min_num)
# break
# print("len_X =", len(X))
# print("len_y =", len(y))
# print("class_counts =", class_counts)
print("errors =", errors)
return X, y
X1, y1 = generate_additional_images(X, y)
show_classes_distribution(y1, 'All Dataset')
# Visualize sample of each class
show_all_classes(X1, y1, num_line=9, num_imgs=3)
# Split Data into Train/Validation/Test
def train_validation_test(X, y):
# Shuffle it
r_idx = np.random.permutation(len(X))
print("len_X =", len(X))
print("len_y =", len(y))
# print("r_idx =", r_idx)
X = X[r_idx]
y = y[r_idx]
# Train/Valid/Test distributions
p_train = 0.6
p_valid = 0.2
p_test = 0.2
# Sanity check
assert p_train + p_valid + p_test == 1.0
# assert p_train + p_valid == 1.0
N = len(X)
train_idx = int(N*p_train)
valid_idx = int(N*(p_train+p_valid))
X_train = X[:train_idx]
y_train = y[:train_idx]
X_valid = X[train_idx:valid_idx]
y_valid = y[train_idx:valid_idx]
X_test = X[valid_idx:]
y_test = y[valid_idx:]
# X_test = X[valid_idx:]
# y_test = y[valid_idx:]
assert len(X_train) + len(X_valid) + len(X_test) == N
assert len(y_train) + len(y_valid) + len(y_test) == N
# assert len(X_train) + len(X_valid) == N
# assert len(y_train) + len(y_valid) == N
return X_train, y_train, X_valid, y_valid, X_test, y_test
# return X_train, y_train, X_valid, y_valid
X_train, y_train, X_valid, y_valid, X_test, y_test = train_validation_test(X1, y1)
# X_train, y_train, X_valid, y_valid = train_validation(X1, y1)
show_classes_distribution(y_train, 'Train + Generated')
show_classes_distribution(y_valid, 'Validation + Generated')
show_classes_distribution(y_test, 'Test + Generated')
In [11]:
# plt.imshow(random_rotate(X[3756]))
In [12]:
# Normalize data
def normalize(data):
pixel_depth = 255
return (data - pixel_depth / 2) / pixel_depth
X_train_norm = normalize(X_train)
X_valid_norm = normalize(X_valid)
X_test_norm = normalize(X_test)
print("Data normalized.")
# print("X_train[0] =", X_valid[0])
Question 2¶
Describe how you set up the training, validation and testing data for your model. If you generated additional data, why?
Answer:
In [13]:
### Define your architecture here.
### Feel free to use as many code cells as needed.
In [14]:
# Config
class SmallConfig(object):
"""Small config. """
# init_scale = 0.1
image_size = 32
num_classes = 43
num_channels = 3
fc1_size = 512
batch_size = 128 # 128
max_epoch = 35 # 250 45 55 - latest
keep_prob = 0.5
lr = 0.01
lr_decay = 0.95
train_size = X_train.shape[0]
In [15]:
# Model Define
class SignsModel(object):
"""Road Sign Classifier"""
def __init__(self, config):
self.batch_size = config.batch_size
self.image_size = config.image_size
self.num_classes = config.num_classes
self.num_channels = config.num_channels
self.train_size = config.train_size
self.config = config
self.add_placeholders()
self.output_logits = self.add_model()
self.loss = self.add_loss_op(self.output_logits)
self.train_op = self.add_training_op(self.loss)
self.prediction = self.add_prediction(self.output_logits)
def add_placeholders(self):
"""Add Placeholders"""
self.input_placeholder = tf.placeholder(tf.float32,
shape=[None, self.image_size, self.image_size, self.num_channels],
name="Inputs")
self.labels_placeholder = tf.placeholder(tf.int64, shape=[None,], name="Labels")
self.dropout_placeholder = tf.placeholder(tf.float32, name="Dropout")
def add_model(self):
"""Model Definition"""
fc1_size = self.config.fc1_size
self.params = {}
conv1_depth = 32
conv2_depth = 64
# Define variables
self.params["conv1_weights"] = conv1_weights = tf.Variable(
tf.truncated_normal([5, 5, self.num_channels, conv1_depth],
stddev=0.1,
dtype=tf.float32), name="conv1_weights")
self.params["conv1_biases"] = conv1_biases = tf.Variable(tf.zeros([conv1_depth], dtype=tf.float32),
name="conv1_biases")
self.params["conv2_weights"] = conv2_weights = tf.Variable(
tf.truncated_normal([5, 5, conv1_depth, conv2_depth],
stddev=0.1,
dtype=tf.float32), name="conv2_weights")
self.params["conv2_biases"] = conv2_biases = tf.Variable(tf.zeros([conv2_depth], dtype=tf.float32),
name="conv2_biases")
self.params["W1"] = W1 = tf.Variable(tf.truncated_normal([self.image_size // 4 * self.image_size // 4 * conv2_depth, self.num_classes]), name="W1")
self.params["b1"] = b1 = tf.Variable(tf.truncated_normal([self.num_classes]), "b1")
# Define ops
# Conv Layer 1
conv = tf.nn.conv2d(self.input_placeholder,
conv1_weights,
strides=[1, 1, 1, 1],
padding='SAME')
relu = tf.nn.relu(tf.nn.bias_add(conv, conv1_biases))
pool = tf.nn.max_pool(relu,
ksize=[1, 2, 2, 1],
strides=[1, 2, 2, 1],
padding='SAME')
# Conv Layer 2
conv = tf.nn.conv2d(pool,
conv2_weights,
strides=[1, 1, 1, 1],
padding='SAME')
relu = tf.nn.relu(tf.nn.bias_add(conv, conv2_biases))
pool = tf.nn.max_pool(relu,
ksize=[1, 2, 2, 1],
strides=[1, 2, 2, 1],
padding='SAME')
# Reshape for fully connected layers
pshape = pool.get_shape().as_list()
pool_line = tf.reshape(pool, [-1, pshape[1] * pshape[2] * pshape[3]])
# Used only during training by setting dropout param to config.keep_prob
pool_line = tf.nn.dropout(pool_line, self.dropout_placeholder)
output = tf.matmul(pool_line, W1) + b1
return output
def add_loss_op(self, output):
"""Loss Op"""
loss = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(
output, self.labels_placeholder))
# Regularization
regularizers = tf.nn.l2_loss(self.params['W1']) + tf.nn.l2_loss(self.params['b1'])
loss += 5e-4 * regularizers
return loss
def add_training_op(self, loss):
"""Add Training Op"""
self.global_step = tf.Variable(0, dtype=tf.float32, name="global_step")
self.learning_rate = tf.train.exponential_decay(
self.config.lr,
self.global_step * self.config.batch_size,
self.train_size, # Train size
self.config.lr_decay,
staircase=True
)
optimizer = tf.train.AdamOptimizer(self.learning_rate)
train_op = optimizer.minimize(loss, global_step=self.global_step)
return train_op
def add_prediction(self, output):
prediction = tf.nn.softmax(output)
return prediction
def eval_predictions(self, session, data):
batch_size = self.config.batch_size
num_classes = self.config.num_classes
size = data.shape[0]
if size < batch_size:
feed_dict = {
self.input_placeholder: data,
self.dropout_placeholder: 1.0
}
return session.run(self.prediction, feed_dict=feed_dict)
predictions = np.ndarray(shape=(size, num_classes), dtype=np.float32)
steps_range = tnrange(0, size, batch_size, desc="Eval pred") # tqdm progress
for begin in steps_range:
end = begin + batch_size
if end <= size:
feed_dict = {
self.input_placeholder: data[begin:end],
self.dropout_placeholder: 1.0
}
predictions[begin:end, :] = session.run(self.prediction,
feed_dict=feed_dict)
else:
feed_dict = {
self.input_placeholder: data[-batch_size:],
self.dropout_placeholder: 1.0
}
batch_predictions = session.run(self.prediction,
feed_dict=feed_dict)
predictions[begin:] = batch_predictions[begin - size]
return predictions
def eval_prediction_labels(self, session, data):
predictions = self.eval_predictions(session, data)
return np.argmax(predictions, axis=1)
def eval_prediction_labels_k(self, session, data, k=1):
predictions = self.eval_predictions(session, data)
values, indices = session.run(tf.nn.top_k(predictions, k=k))
return values, indices
def eval_accuracy(self, session, data, labels):
predictions = self.eval_predictions(session, data)
sum_ones = np.sum(np.argmax(predictions, axis=1) == labels)
print("sum_ones =", sum_ones, " of len =", len(predictions))
accuracy = sum_ones / len(predictions)
return accuracy
def run_epoch(self, session, data, labels, train_op = None, verbose = None):
config = self.config
batch_size = config.batch_size
keep_prob = config.keep_prob
num_steps = len(data) // batch_size
if not train_op:
train_op = tf.no_op()
keep_prob = 1.0
print("keep_prob =", keep_prob)
steps_range = tnrange(num_steps, desc="Eval") # tqdm progress
else:
steps_range = tnrange(num_steps, desc="Steps") # tqdm progress
total_loss = []
for step in steps_range:
x = data[step*batch_size:(step+1)*batch_size]
y = labels[step*batch_size:(step+1)*batch_size]
feed_dict = {
self.input_placeholder: x,
self.labels_placeholder: y,
self.dropout_placeholder: keep_prob
}
_, loss, prediction = session.run([train_op, self.loss, self.prediction], feed_dict=feed_dict)
# print("- conv =", conv)
# print("- loss =", loss)
# print("- logits =", logits)
# print("- prediction =", prediction)
# print("- prediction1 =", prediction[:, np.argmax(prediction, axis=1)])
# print("- prediction max =", np.argmax(prediction, axis=1))
# print("- y =", y)
total_loss.append(loss)
# print("loss =", loss)
if verbose is not None and (step + 1) % verbose == 0:
print("%d/%d loss = %f" % (step, num_steps, np.mean(total_loss)))
# if step > 80: break
return np.mean(total_loss)
def train_model(self, session, train_data, train_labels, valid_data, valid_labels):
max_epoch = self.config.max_epoch
total_train_loss = []
total_valid_loss = []
epoch_valid_loss = 0
for epoch in tnrange(max_epoch, desc="Epochs"):
epoch_train_loss = self.run_epoch(session, train_data, train_labels, train_op = self.train_op, verbose = 20)
total_train_loss.append(epoch_train_loss)
epoch_valid_loss = self.run_epoch(session, valid_data, valid_labels)
total_valid_loss.append(epoch_valid_loss)
if epoch % 1 == 0 or epoch + 1 == max_epoch:
print("Epoch %d/%d: " % (epoch + 1, max_epoch))
print("Loss: %f" % epoch_train_loss)
print("Validation Loss = %f" % (epoch_valid_loss))
lr = session.run(self.learning_rate)
print("LR =", lr)
x_lab = np.arange(max_epoch)
fig, ax = plt.subplots()
ax.plot(x_lab, total_train_loss, '', label='Train Loss')
ax.plot(x_lab, total_valid_loss, '', label='Valid Loss')
ax.legend(loc='upper right')
ax.set_title("Loss over Epochs")
plt.show()
return epoch_valid_loss
print("X_train_norm.shape =", X_train_norm.shape)
print("y_train_norm.shape =", y_train.shape)
print("X_valid_norm.shape =", X_valid_norm.shape)
print("y_valid_norm.shape =", y_valid.shape)
print("X_test_norm.shape =", X_test_norm.shape)
print("y_test_norm.shape =", y_test.shape)
# Reset graph so to have each time the same variable (useful for store/restore)
tf.reset_default_graph()
config = SmallConfig()
model = SignsModel(config)
# To train model from scratch change this to False
restore = False
restore_file = 'params/model_params_201611241412.ckpt'
# Create a session
session = tf.Session()
saver = tf.train.Saver(write_version=1)
# Restore or Initialize params
if restore and restore_file:
saver.restore(session, restore_file)
print("Model restored from ", restore_file)
else:
# Init new
all_variables = tf.all_variables()
init = tf.initialize_variables(all_variables)
session.run(init)
print("Initialized new model params.")
print("Train new model ...")
model.train_model(session, X_train_norm, y_train, X_valid_norm, y_valid)
save_time = time.strftime("%Y%m%d%H%M")
saved_path = saver.save(session, "model_params_%s.ckpt" % save_time)
print("Model params saved to ", saved_path)
print("Checking model test accuracy:")
accuracy = model.eval_accuracy(session, X_test_norm, y_test)
print("Test Accuracy = %.4f" % accuracy)
In [16]:
# Test Accuracies
print("Checking test accuracy:")
accuracy = model.eval_accuracy(session, X_test_norm, y_test)
print("Test Accuracy = %.4f" % accuracy)
print("Checking valid accuracy:")
accuracy = model.eval_accuracy(session, X_valid_norm, y_valid)
print("Valid Accuracy = %.4f" % accuracy)
print("Checking train accuracy:")
accuracy = model.eval_accuracy(session, X_train_norm, y_train)
print("Train Accuracy = %.4f" % accuracy)
Question 3¶
What does your final architecture look like? (Type of model, layers, sizes, connectivity, etc.) For reference on how to build a deep neural network using TensorFlow, see Deep Neural Network in TensorFlow from the classroom.
Answer:
In [17]:
### Train your model here.
### Feel free to use as many code cells as needed.
In [18]:
# Confusion Matrix Evaluation
def confusion_matrix(labels, pred, num_classes):
cm = np.zeros([num_classes, num_classes])
for l,p in zip(labels, pred):
cm[l,p] += 1
s = np.sum(cm, axis=1)
r = cm / s.reshape([len(s),1])
return r
# Borrowed from http://scikit-learn.org/stable/auto_examples/model_selection/plot_confusion_matrix.html#sphx-glr-auto-examples-model-selection-plot-confusion-matrix-py
def plot_confusion_matrix(cm, classes,
normalize=True,
title='Confusion matrix',
cmap=plt.cm.Blues):
"""
This function prints and plots the confusion matrix.
"""
if normalize:
cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]
plt.imshow(cm, interpolation='nearest', cmap=cmap)
plt.title(title)
plt.colorbar()
tick_marks = np.arange(len(classes))
plt.xticks(tick_marks, classes, rotation=45)
plt.yticks(tick_marks, classes)
thresh = cm.max() / 2.
for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):
plt.text(j, i, "%.2f" % cm[i, j],
horizontalalignment="center",
color="white" if cm[i, j] > thresh else "black")
plt.tight_layout()
plt.ylabel('True label')
plt.xlabel('Predicted label')
prediction_labels = model.eval_prediction_labels(session, X_test_norm)
print("prediction_labels =", prediction_labels)
cm = confusion_matrix(y_test, prediction_labels, model.config.num_classes)
plt.figure(figsize=(22,22))
plot_confusion_matrix(cm, classes=range(model.config.num_classes),
title='Confusion matrix')
In [19]:
# plt.figure(figsize=(22,22))
print("len =", len(y_test))
print("matches =", np.sum(prediction_labels == y_test))
# plot_confusion_matrix(cm, classes=range(model.config.num_classes),
# title='Confusion matrix')
In [20]:
# print(prediction_labels[:20])
# print(y_test[:20])
Question 4¶
How did you train your model? (Type of optimizer, batch size, epochs, hyperparameters, etc.)
Answer:
Question 5¶
What approach did you take in coming up with a solution to this problem?
Answer:
Step 3: Test a Model on New Images¶
Take several pictures of traffic signs that you find on the web or around you (at least five), and run them through your classifier on your computer to produce example results. The classifier might not recognize some local signs but it could prove interesting nonetheless.
You may find signnames.csv useful as it contains mappings from the class id (integer) to the actual sign name.
Implementation¶
Use the code cell (or multiple code cells, if necessary) to implement the first step of your project. Once you have completed your implementation and are satisfied with the results, be sure to thoroughly answer the questions that follow.
In [21]:
### Load the images and plot them here.
### Feel free to use as many code cells as needed.
def resize_image(img):
im = cv2.resize(img, (32,32))
im = cv2.cvtColor(im, cv2.COLOR_BGR2RGB)
return im
def load_images_from_folder(folder):
images = []
for filename in os.listdir(folder):
img = cv2.imread(os.path.join(folder,filename))
if img is not None:
images.append(resize_image(img))
return np.array(images)
# Load images from folder
imgs = load_images_from_folder('test_imgs')
imgs_norm = normalize(imgs)
vals, imgs_pred_labels = model.eval_prediction_labels_k(session, imgs_norm, k=5)
# print("imgs_pred_labels =", imgs_pred_labels)
# print("vals =", vals)
num = 0
for idx, val, im in zip(imgs_pred_labels, vals, imgs):
num += 1
plt.figure(figsize=(2,2))
plt.imshow(im)
plt.title(num)
text = []
for i,v in zip(idx, val):
text.append("%.5f - %s\n" % (v, signnames[i]))
plt.text(35, 32, "".join(text))
Question 6¶
Choose five candidate images of traffic signs and provide them in the report. Are there any particular qualities of the image(s) that might make classification difficult? It would be helpful to plot the images in the notebook.
Answer:
In [22]:
### Run the predictions here.
### Feel free to use as many code cells as needed.
Question 7¶
Is your model able to perform equally well on captured pictures or a live camera stream when compared to testing on the dataset?
Answer:
In [23]:
### Visualize the softmax probabilities here.
### Feel free to use as many code cells as needed.
Question 8¶
Use the model's softmax probabilities to visualize the certainty of its predictions, tf.nn.top_k could prove helpful here. Which predictions is the model certain of? Uncertain? If the model was incorrect in its initial prediction, does the correct prediction appear in the top k? (k should be 5 at most)
Answer:
Question 9¶
If necessary, provide documentation for how an interface was built for your model to load and classify newly-acquired images.
Answer:
Note: Once you have completed all of the code implementations and successfully answered each question above, you may finalize your work by exporting the iPython Notebook as an HTML document. You can do this by using the menu above and navigating to \n", "File -> Download as -> HTML (.html). Include the finished document along with this notebook as your submission.
In [ ]: